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NONDEGENERATE SECOND QUADRATIC FORM 
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Abstract. Consider a closed connected hypersurface in M n with constant 
signature (k, I) of the second quadratic form, and approaching a quadratic 
cone at infinity. This hypersurface divides R n into two pieces. We prove 
that one of them contains a fc-dimensional subspace, and another contains a 
Z-dimensional subspace, thus proving an affine version of Arnold hypothesis. 
We construct an example of a surface of negative curvature in R 3 with slightly 
different asymptotical behavior for which the previous claim is wrong. 
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1. Introduction 

In this paper we prove three results connected to an Arnold hypothesis formu- 
lated in [JlJ. Recall first this hypothesis. 

Definition 1. A smooth hypersurface in MP" +1 is called (k, ^-hyperbolic if its sec- 
ond quadratic form has signature (k, I). In other words, in some affine system of co- 
ordinates (x, y, z) its local equation is z = 53i=i x 1 ~ Ylj—i vl + higher order terms. 
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One class of examples of such hypersurfaces is the class of quadrics. Namely, 
let Q(x, y) be a non-degenerate symmetric bilinear form in of signature {k + 

1,1+1) (i.e. its restriction to some [k + l)-dimensional subspace L + is positively 
defined, and its restriction to some (I + l)-dimensional subspace L_ is negatively 
defined, and k + I = n — 1). Then a hypersurface Sq in M_P" given by equation 
Q(x,x) = is smooth and (fc, ^-hyperbolic. 

The hypersurface Sq has the following remarkable property: the domain bounded 
by Sq (i.e. the domain {Q(x, x) ^ 0}) contains a Z-dimensional projective subspace 
P(L + ) - a projectivization of L + , and its complement contains a fc-dimensional 
projective subspace P(L_) - a projectivization of L_ . 

Arnold's hypothesis claims existence of such subspaces for any (k, i)-hyperbolic 
hypersurface. 

Conjecture 1 (Arnold Conjecture). 1. For any domain U C RP n bounded by a 
connected smooth (k, I) -hyperbolic hypersurface B there exist a projective subspace 
L- of dimension k not intersecting U and a projective subspace L+ of dimension I 
contained in U . 

2. Any projective line joining a point of L + and a point L- intersects B at exactly 
2 points. 

Apart from the case of quadrics discussed above, another fact justifying this 
conjecture is the following well-known fact. Consider a locally convex connected 
surfaces S in RP" . Then S is a boundary of some domain in RP™ and this domain 
doesn't intersects some hyperplane, see 0. This case corresponds to the case of 
k — or / = of the Arnold Conjecture. 

We prove in [0 the first case k = I = 1 of the Arnold Conjecture in some addi- 
tional assumptions. Namely, we consider a smaller then class of (1, l)-hyperbolic 
surfaces class of projective L-convex-concave subsets of M.P 3 . This class is a close 



relative of the class convex-concave sets considered in §5.1. We prove that any 
i-convex-concave subset contains a line inside. 

In this paper we deal with affine version of Arnold's hypothesis. Namely, we 
consider (k— 1, ^-hyperbolic hypersurfaces in M. k+l with some prescribed asymptotic 
behavior at infinity. Our results can be roughly summarized as follows: if the 
asymptotic at infinity forces the closure in M.P k+l of the hypersurface to be (fc— 1, l)- 
hyperbolic, then the domain bounded by the hypersurface contains a line. And if 
the closure is not (k — 1, Z)-hyperbolic, then one can construct a domain bounded 
by such hypersurface and not containing any line inside. 

Here is more exact description of the results. Consider (fc — 1, Z)-hyperbolic 
hypersurfaces M in R™, k + I = n. We say that M approaches a surfac e L at 



infinity if M and L are arbitrarily C 2 -close outside a big enough ball (see § |3.1| for 

</=l X k+jl 



an exact e, (5-definition). For example, the quadric x i ~ zCi^i^fe+i} = 



approaches the cone K = {X}j=i x i — x k+j} c ^" a * infinity for any A. 

We prove the following theorem: 

Theorem 1. The first claim of the Arnold hypothesis is true for any (fc — 1,1) 

r 2 



hyperbolic hypersurface M approaching a quadratic cone K = {X)i=i x \ = ^2\=i x k+j} c - 



at infinity. 



We also prove the following result, strengthening the second part of the Arnold 
Conjecture. 
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Theorem 2. Any (1, 1) -hyperbolic surface M in R 3 approaching a quadratic cone 
K = {x 2 + y 2 = z 2 } intersects any ray from the origin in at most one point. In 
other words, the projection M — » § 2 = {x 2 + y 2 + z 2 = 1} is embedding. 

Note that the surface described in Theorem |l| has a C 1 -closure in IP™. Consider 
the simplest case k = I = 1, and denote K- = KP\{z ^ 0} and K + = KP\{Z ^ 0}. 
Will the result remain true if we consider surface B approaching union of translated 
K- and K + ? It turns out that if the translates intersect, then the domain bounded 
by the surface B still contains a line, and if the intersection is empty, then this 
is not necessarily true. Note the different behavior of the projective closure B of 
the surface in these two cases. In both cases the points of B \ B are not smooth 
points of the closure. However, in the first case the B is (1, l)-hyperbolic after 
arbitrarily small perturbation, and in the second case the surface B is locally convex 
at these points. This coincides with what the Arnold Conjecture prescribes, further 
strengthening it. 

Theorem 3. Let K' = {(x,y,z) x 2 + y 2 = (\z\ — l) 2 , \z\ ^ 1} be a union 
of non-intersecting translates of K- and K + . There are (1,1) -hyperbolic surfaces 
approaching K' at infinity and not containing any lines. 

All these result can be considered in more general context of existence and prop- 
erties of a solution of some boundary problem. A natural boundary problem is to 
find a compact smooth (1, l)-hyperbolic surface which boundary is a given tuple of 
non-intersecting closed smooth curves and which is tangent at this boundary to the 
given set of planes tangent to the curves. The Theorem [l] follows from the fact that 
solution of some boundary problem of this type cannot intersect an open domain - 
the interior of the cone K. 

2. Preliminaries: Quadrics 

The hypersurface M of the Theorem ^ is approaching at infinity a cone given by 
a quadric. In this paragraph we collect some standard facts about quadrics we will 
need later. 

Theorem 4. Let Q(x) — X)i=i x 1 ~ Ylj=i x l+j' an ^ ^ Qe = {/ = e i be ^ s ^evel 
hyper surf aces. 

1. restriction of Q to a tangent plane T x Q e C R fe+i at point x £ Q(£) is a 
polynomial of second degree having signatures (k, I — I), (k—1, 1—1) or (k— 1, 1) 
according to e < 0, e = or e > correspondingly; 

2. Q e is (fc, I — 1) -hyperbolic if e < and (k — 1, l)-hyperbolic if e > 0; 

3. Projectivization of Qo is a (k — 1, 1 — 1) -hyperbolic hypersurface in M.P + . 

Proof. First we prove a general Lemma. To a nondegenerate bilinear form q on a 
linear space L corresponds an isomorphism q between L and its dual L* , defined 
by the condition £(x) — q(q(£),x) for I S L* and all x G L. Using this form one 
can define a bilinear form g*(£i,£ 2 ) = q(q(li), qih j) on L* . 

Lemma 1. Restriction of q on a hyperplane H = {£ = 0} C L has signature 
(k, l-l), {k- 1,1-1), or (k - 1, 1) if q*(£, i) < 0, q*(£, i) = or q*(£, i) > 
correspondingly. 

Proof. By definition q(q(£),x) — is equivalent to x G H. So q*(£,£) = means 
y = q(£) G H , i.e. the restriction of q on H is degenerate. From the other hand, 
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H should intersect subspaces where q is positive/negative definite by subspaces of 
dimensions k — 1 and I — 1 at least, so the only possible signature is (k — 1, 1 — 1). 
Vice versa, if for some y S H and for all x £ H we have q(y, x) = 0, then £ is 
proportional to q~ 1 (y) and so q*(£,£) = q{y,y) = 0. 

So suppose £) 7^ 0. Then Gramm-Schmidt procedure starting from y — q{£) 
is possible and results in a basis which first vector is y and all the rest span a 
hyperplane of vectors g-orthogonal to y, i.e. H . Since in this basis q is diagonal, 
we easily see that the signature of the restriction of q to H is as required. □ 

To prove the Theorem || apply Lemma [j] to Q(x) — q(x, x) in R™, where n = k+l. 
The first claim of the Theorem [| follows from the Lemma |l| and the fact that 
q(dQ(x)) = 2x, so q*(dQ(x),dQ(x)) = iq(x,x) = 4e. 

To prove the second claim of the Theorem [| change coordinates in such a way that 
x 6 Q(s) is an origin and T x Qie) = {x n = 0}. The restriction Qi of Q to T x Qie) 
is a quadratic polynomial, without linear and free terms in these coordinates. So 
Q = e + \x n + Qi(xi, x n -\) + x n i{x\, x n -i), where Qi is a homogeneous 
polynomial of degree 2 and £(x\, x n -i) is linear. An easy computation shows 
that x n = —X~ l Qi{xi, ...,x n -i) + ... on {Q = e}. The second claim follows now 
from the first claim of the Theorem ||. 

The third claim follows since the kernel of the bilinear form qi on T x Q/e) 
corresponding to the quadratic form Qi is exactly the kernel of the projection 
R n -> MP™" 1 . 

□ 

Denote by § fc +^ 1 the standard sphere {Et*=i x i +Ej=i x l+ 3 = 1 ] C ^ k+l ■ The 
standard scalar product {x,x') — J2i x i x 'i m L defines, as above, a scalar product 
in L* and also an isomorphism of L and L*. For a smooth cooriented hypersurfacc 
M C R" the Gauss map G : M — » S" -1 maps a point xeMto the vector normal 
to M at x. If M = {P = 0} and dP ^ on M, then the Gauss map is a composition 
of the map x — > -p^p- and the isomorphism between K™ and (R")* provided by a 
standard scalar product. A classical computation shows that the Jacobian of the 
Gauss map is exactly the Gaussian curvature of the hypersurfacc. 

Lemma 2. Let Q = Ej=i x l ~ E'=i x k+j be as in Theorem 0. Then the pull-back 
of Q* from (R™)* to R" by the isomorphism given by the standard scalar product 
coincides with Q. The Gauss map provides diffeomorphisms between {Q — e > 0} 
and S n n {Q > 0}, between {Q = s < 0} and § n n {Q < 0}, and maps §" n {Q = 0} 
diffeomorphically onto itself 

This follows immediately from the explicit formulae for the Gauss mapping G of 
the quadric {Q = e}. Namely, 

G(xi, ...,X k ,X k+ i, ...,X k +l) = j—r-{xi, ...,Xk,~Xk+l, -x k +i). 

11*11 

□ 

The topology of the domains, and therefore of level hypersurfaces of Q is quite 
simple. 

Corollary 1. {Q = e > 0} is diffeomorphic to § fc-1 x B l , {Q = e < 0} is diffeo- 
morphic to S^ 1 x B k and S™ H {Q = 0} is diffeomorphic to S*^ 1 x S^ 1 

For example, S n n {Q - 0} - {^ti *i = E}=i 4 +j = 5} = x 
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3. Surface doesn't enter the half-cones 

3.1. Exact formulation of the result. In this section we prove the Theorem [p. 
We start with a definition of an affine version of (fc, ^)-hyperbolicity and state more 
precisely the asymptotic conditions on the hypersurface M . 

Definition 2. A smooth connected hypersurface M lying in K" equipped with a 
standard Euclidean metric is (k, ^-hyperbolic if its second quadratic form is every- 
where nondegenerate and have signature (k,l). 

Definition 3. We say that a hypersurface M approaches a hypersurface L at in- 
finity if for any e > there exists an B > such that 

1. there exists a diffcomorphism <f> : L \ Br — >• M\ Br, such that \\4>{x) — x\\ < e 
for any x G L \ B^r and 

2. there is a diffeomorphism -0 of the Gauss images of L n S^r 1 on to the Gauss 
image of (j>{M n S^r 1 ) c ^ sucn that dist(V'Oc), x) < e in standard metric on 
S"^ 1 , where S^fl 1 denotes a sphere of radius 2R with center at the origin. 

The proof of the Theorem [j] goes as follows. First, using simple topological ar- 
guments, we prove that image under the Gauss map of the hypersurface M does 
not intersect the image under the Gauss map of the quadric {Q = e > 0} for any 
positive e. Second, we note that the interior U = {Q > 0} of the cone K can be 
exhausted by these quadrics. If we suppose that M n U is nonempty and compact, 
then an analogue of the Rolle theorem says that there should be a level curve of Q 
lying in U and tangent to M , namely the level curve {Q = max l£ M Q(x)}. This 
contradicts to the properties of the Gauss map of M mentioned above. The as- 
sumption of compactness can be dropped by a slight modification of this arguments. 

3.2. The topological Lemma. The proof of the Theorem [l] starts from a simple 
topological lemma. This lemma will be applied further to the Gauss map of (k, l)- 
hyperbolic hypersurfaces. 

Lemma 3. Let M be a compact connected manifold with boundary dM and f : 
M — > N be a local diffeomorphism to a compact simply- connected manifold N , 
ni(N) = 0. Suppose that the restriction f\dM of f to each connected component 
of the boundary of M is also a diffeomorphism (i.e. also embedding). Then f is a 
diffeomorphism of M to f(M). 

Proof. An easy case is dM = 0. In this case / is a covering, so should be a trivial 
one (since wi(N), being trivial, has no nontrivial subgroups). 

The general case will be reduced to this case by gluing "hats" to M, thus elimi- 
nating the boundary components one-by-one. 

Namely, consider a connected component of dM (denote it by B). Its image 
f(B) is a cooriented hypersurface in N . Indeed, f(B) divides any sufficiently small 
neighborhood of any its point f(b) into two parts, and one of them is distinguished, 
since it is an image of a small neighborhood of b in M. Therefore iV\ f(B) consists 
of two open parts (having even and odd number of preimages correspondingly). 
Call the part which doesn't intersect the image of a sufficiently small neighborhood 
of B by "hat". We can glue the "hat" to M along B = f(B): the new manifold 
will be the union of B and the " hat" with the neighborhood of b £ B being defined 
as union of the connected component of / _1 ([/) containing b and the intersection 
of U and the "hat" (where U C N is a small open ball containing /(&)). 
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Repeating this operation with all components of the boundary, we get a new 
manifold M without a boundary and M C M. The map / extends to / : M — > N 
in a trivial manner: we define it to be an identity on "hats". The map / satisfies 
conditions of the Lemma [| so it is a global diffeomorphism by the first part of the 
proof. So / is also a diffeomorphism, since / = /\m- d 

Corollary 2. Let M be a compact connected oriented (n — 1) -dimensional sub- 
manifold with boundary of R™ , and suppose that its second fundamental form is 
everywhere nondegenerate, including the boundary. Assume that the restriction of 
the Gauss map of M to each connected component of the boundary DM is one-to- 
one. Then the Gauss map of M itself is one-to-one. 

3.3. Gauss image of (k — 1, ^-hyperbolic hypersurface. Consider a (k — 1, l)- 
hyperbolic connected closed hypersurface M C R fc+ ' +1 approaching the quadratic 
cone K = {Q = 0} at infinity, where, as before, Q(x) — Yli=i x \ ~ Ylj=i x k+j * s a 
quadratic form on R™, n = k + l. Similarly to the previous Corollary, we prove that 
the Gauss image of M coincides with the Gauss image of a (k — 1,1)- hyperbolic 
level hypersurface of Q. 

We will consider later the case of a surface M approaching at infinity the cone 
K' = {{\z\ - l) 2 = x 2 + y 2 } C M 3 (i.e. k - 1 = I = 1). The Lemma | is true for 
this case as well. 

Lemma 4. Let M be a [k— 1, l)-hyperbolic closed connected hypersurface approach- 
ing K . Then the Gauss mapping maps M diffeomorphically onto the Gauss image 
of a (k — 1, 1) -hyperbolic level hypersurface of Q. 

Proof. The Jacobian of the Gauss map is equal to the Gaussian curvature, i.e is 
non- vanishing. Therefore the Gauss map is a local diffeomorphism. M is not a 
compact manifold with boundary, so the Lemma |^ is not directly applicable. 

Consider the compact Mr = M n Br, where Br C W 1 is a big closed ball of 
radius R centered at the origin, and denote by 8Mr the boundary of Mr. Its 
image G(8Mr) under the Gauss map C 1 -converge, as R — * oo, to the Gauss image 
of K n 8Br, which coincides with the Gauss image of the whole cone K. 

Lemma 5. G{M R ) n G{dM R >) = for any R and any R' > R. 

For big enough R' this follows immediately from Lemma ^ applied to Mr* and 
its Gauss map. The Lemma ^ is applicable since Mr* is compact and the restric- 
tion of the Gauss map to its boundary is a diffeomorphism, due to the condition 
" approaching at infinity" . Therefore this is true for any R. □ 

Corollary 3. The Gauss image of Mr doesn't intersect the G(K). 

Indeed, if the intersection is non-empty, then there is a point of G(k) which 
lies in the interior of G(Mr'), where R' is any number greater than R. Therefore 
this point cannot be a limit point of G(8Mr) as R — * oo. This contradicts to the 
condition that M approaches K at infinity. □ 

Therefore the G(Mr) should lie entirely in one of the connected components 
into which the G(K) divides the sphere S™ -1 . Since 8G(Mr) converge uniformly 
to G(K), we conclude that G(M) is exactly one of them and the Gauss mapping 
is a diffeomorphism. 

So the last part is to prove that G{M) fall into the right connected component. 
If k = I, then all nonsingular level hypersurfaces of Q are (k — 1, Z)-hyperbolic 
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(since (k — 1, Z)-hyperbolicity and (k,l — l)-hyperbolicity are the same), and there 
is nothing to prove. So we suppose that k ^ I. Then the Gauss image of K divides 
the sphere into two domains of different topological type: the Gauss image of 
a (k — 1, Z)-hyperbolic level surface is diffcomorphic to S fe_1 x B l , and the Gauss 
image of a (k,l — l)-hyperbolic level hypersurface is diffeomorphic to §' _1 x B k . 
Denote these domains by D + and D_ correspondingly so that D + = G({Q = +1}) 
and D_ = G{{Q = —1}). Since the Gauss mapping of M is a diffeomorphism, we 
get that M is diffeomorphic to D + or Z)_, and our goal is to exclude the last 
possibility. 

We will prove that M is topologically different from D_. We will apply the 
Morse theory to M and the restriction of the linear functional f — x u \m to M. 

Since V(x n ) = e n = (0, 0, 0, 1) G Z)_, the function / has exactly two critical 
points on M, namely the preimages of — e„ and e„ under the Gauss map of M 
(which is a diffeomorphism). The condition of (k — 1, Z)-hyperbolicity means that 
both critical points are non-degenerate and their indices are equal to k — 1 or I. 
Let R be a big number and denote by M — M n {x n < — R}. Although the level 
sets {/ = c} = M n {x n = c} are not compact, their behavior at infinity is trivial. 
Indeed, it the same as of the sections of cone K by the hyperplanes {x n — c}, due 
to the condition at infinity. Therefore the standard results of the Morse theory still 
hold, so the dimension hi(M,M) of the group of relative homologies Hi(M,M) 
is less than the number of critical points of index i of the function /. The pair 
(M,M) is diffcomorphic (through the Gauss maps) to the pair (<5_i,Q_i), where 
Q-i = {Q = -1}, and QTi = Q_i n {x n < -R}, so Hi (M, M = H^Q^Q^x). 
The latter can be easily computed to be equal to I for i = k (since Q-i/Q-i = 
gi-i v § fe ), and this contradicts to the fact that / has no critical points of index k 
(recall that k^l). 

□ 

3.4. Rolle Lemma. Let Q(x) = x\ + ... + x\ — x\ +1 — — x 2 n be a quadratic form 
in W\ The cone K = {Q = 0} divides M™ into two parts, {Q > 0} and {Q < 0}. 
We prove in this paragraph that M does not intersect one of these domains. In fact 
the assumption are weaker than before. 

Theorem 5. Let M be a smooth connected hypersuface M such that dist(x, K) — > 
as M 3 x — > oo. Suppose that the Gauss image of M is disjoint from the Gauss 
image of {Q = —1}. Then M does not intersect the whole domain {Q < 0}. 

The proof is a specialization of the following general lemma. 

Lemma 6. Let M C R" be a smooth closed embedded hypersurface without bound- 
ary and suppose that its image under the Gauss mapping Gm ■ M — > §™ _1 does 
not intersect a domain U C S n_1 . Suppose that U is symmetric with respect to 
antipodal map x — ► —x of S" . 

Suppose that on R™ we are given a function f with nonnegative only critical 
values. Suppose that ||v/(a)|| e ^ < Cm other words, G({f = t} C U for 

any t < 0). If f(x) — > as M 3 x — > oo, then f is nonnegative on M. 

Proof. Suppose that f{x) < for some x £ M. Let xq G M the point of minimum 
of the restriction of / to M. It exists since M n {/ ^ f{M)} is compact and 
nonempty, xq is a critical point of the restriction of / to M. Equivalently, T Xo M is 
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perpendicular to the nonzero vector V/(xo). This means that -Ts(xo) G U, which 
is forbidden. □ 

Apply this Lemma to the proof of the theorem. A first candidate for the function 
/ is the Q itself: Gauss images of {Q = t < 0} are all equal and do not intersect the 
Gauss image of M. However, Q itself do not satisfy the conditions of the Lemma [| 
one should slightly adjust Q to ensure that the restriction of / to M tends to zero 
at infinity. 



Denote \Jx\ + .. + x\ by a and J x 2 +1 + ... + x\ by b. Suppose that Q(x) — 

a 2 — b 2 take a negative value — 2e at some point x' E M, i.e. that M intersects 
the domain {Q < — e}. Consider the function fi — \J a 2 + e — b and denote by / 
its smoothening: the /i is not smooth at b = 0, but one can smoothen fx without 
changing it on {/i < 0} = {Q < — e}, the only domain interesting for us. The main 
point is that the Gauss image of the negative level hypersurfaces of / is the same 
as the Gauss image of {Q = —e}. This is most evident in the planar k = I = 1 
case, where the level curves of / are just translates of {Q — — e}. Indeed, take 
a point (x\, x n ) such that t = f(x) < 0. A simple computation shows that, 
first, df(xi, ...,x n ) is proportional to dQ(xi, Xf-, Xxt+i , Xx n ) for A = ^* anc | ) 
second, Q(x\, x&, Aajfe+i, \x n ) = —e. This computation proves that the Gauss 
images of {/ = t < 0} and {Q — —e} are equal, and also shows that / has only 
positive critical values. 

One can easily see that different level curves of / lie on positive distance one 
from another, so the only one approaching K at infinity is the zero level curve 
{/ = 0} = {Q = — e}. Therefore, since dist(x,K) — > as M 9 x — > oo, the 
restriction of / to M tends to zero at infinity. 

So / satisfies the conditions of the Lemma ^|, and therefore M n {Q < — e} = 
Mn{/<O} = 0,a contradiction with the choice of x' . □ 

3.5. End of the proof of the Theorem |l|. The rest of the proof of the Theorem!^ 
is just application of the two results proven above, the Lemma |] and the Theorem |f 

First prove existence of an /-dimensional subspace in one of the domains into 
which M divides R". Suppose first that k ^ I. In this case the quadrics Q\ = 
{Q = 1} and Q_i = {Q = —1} have different signatures of the second quadratic 
forms: the first one is (k — 1, Z)-hyperbolic, and the second is (k, I — l)-hyperbolic. 
By Lemma [|the Gauss image of the (k — 1, /)-hyperbolic hypersurface M coincide 
with the Gauss image of Qi, and is therefore disjoint from the Gauss image of Q-\. 
So, by Theorem ^, M does not intersect the domain {Q < 0}, which contains the 
/-dimensional subspace {x\ = ... = x^ =0}. 

If k = I, then both Q\ and Q_i are (k — 1, Z)-hyperbolic, and Lemma |] claims 
that the Gauss image of M coincides with the Gauss image of one of them. Taking 
— Q instead of Q if necessary, we can assume that G(M) coincides with G(Q±), and 
the same arguments hold. 

The existence of a (k — l)-dimensional affine subspace in the second part of 
W 1 \ M is evident. By assumption M approaches the cone K at infinity, so the 
distance between M \ Br and K \ Br is less than distance between K \ Br and 
L = {xk+\ = ... = x n = 0} for big enough ball Br. So take any (k— l)-dimensional 
affine subspace of L lying outside Br, and it will not intersect M. 
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4. Projection from the origin 

Starting from this moment we will deal with (1, l)-hyperbolic surfaces in R 3 only. 
So we will omit the (1, 1) and will call (1, l)-hyperbolic surfaces hyberbolic surfaces. 

The Theorem | ensures that a hyperbolic surface approaching the cone K = 
{x 2 + y 2 = z 2 } at infinity do not intersect any line passing through the origin and 
lying in the domain {x 2 + y 2 < z 2 }. We prove here that any line passing through 
the origin intersects M at at most two points. 

Theorem 6. Let M C R 3 be a hyperbolic surface approaching the standard cone 
K at infinity. 

Then the restriction to M of the projection ir : R 3 — > § 2 = {\\x\\ = 1} is 
embedding. 

4.1. Arnold's formula. The Theorem || follows from a remarkable formula due 
to Arnold, see jjj. Consider a generic smooth hypersurface M c RP 3 . Denote by 
#{M fl £} number of its points of intersections with a line £ and by sign(M, £) the 
number of point x £ M containing the line £ in their tangent planes counted with 
multiplicities. The multiplicity is equal to " + 1" if the Gaussian curvature of M is 
positive at x and to " — 1" if it is negative at x (if the curvature at x is zero then 
the formula for multiplicity is more complicated). 

Lemma 7 (Arnold, 88). For a generic smooth hypersurface M C RP 3 the sum 
#{M n £} + sign(Af, £) is the same for all I and is equal to the Euler characteristic 
ofM 

Sketch of the proof for a semialgebraic M (due to O.Viro). Take out from M its 
points of intersection with £ and compute the Euler characteristic of the result 
using Fubini theorem for Euler characteristic. Namely, the Euler characteristic of 
M \ £ is equal to the integral over the space of all planes L t , t £ RP 1 , containing 
£ of the Euler characteristic of M t = {M \ £} n L t . For simplicity, suppose that 
each section M t has at most one singular point (if not, perturb £ slightly). Each 
nonsingular section M t is a one-dimensional manifold, so is a union of circles and 
open intervals with ends at removed points. Therefore its Euler characteristic is 
equal to —#{M n £} (since Euler characteristic of a circle is equal to zero). Euler 
characteristic of a singular section differs by +1 or by — 1, depending on the sign of 
the curvature of M at the singular point. Indeed, if the curvature at the singular 
point is negative, then the section has a self-intersection, so the Euler characteristic 
drops by 1. If the curvature at the singular point is positive, then the section has 
an isolated point, and Euler characteristic increases by 1. 

Since the Euler characteristic of MP 1 is equal to zero, the integration of — #{M n 
£} over RP 1 gives zero. So the Euler characteristic of M \ £ , being equal to the 
integral of the Euler characteristic of M t over RP 1 , is equal to sign(M, £), and the 
result follows. □ 

4.2. Compactification of M and end of the proof of the Theorem ^. We 

apply Lemma to the closure of M in RP 3 . First, we have to show that the closure 
of M in RP 3 is a smooth surface. 

Lemma 8. The closure M of M in RP 3 is smooth. 

Proof. Take affine coordinates x = £,y = -,w = ^. We are interested in the 
points of M fl {w — 0}. The first part of the condition "M approaches K at 
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infinity" implies that M approaches {x 2 + y 2 = 1} faster than \w\, so M is smooth 
at these points. The second part means that as x G M tends to xo G M n {w = 0} 
the limit of tangent planes T X M exists and is equal to the tangent plane at xq. This 
means C 1 -smoothness of M. □ 

For the hyperbolic surface M the curvature is always negative. Therefore the sign 
in Lemma [?] is always " — " . The Euler characteristic of M is equal to the Euler 
characteristic of {x 2 + y 2 = z 2 + w 2 } (essentially Lemma ||), i.e. is equal to zero. 
So the Lemma claims in this case that for generic £ 

#{M n(} = #{x G M\£c T X M). (4.1) 

We want to prove that projection of M to S 2 has no folds, i.e. is a local diffeo- 

morphism. In other words, we have to show that no tangent plane to M passes 

through the vertex O of the cone. Suppose otherwise and take a plane tangent to 

M and passing through O. The normal to this plane lies in the image of M under 

the Gauss map, i.e. in § 2 n {\z\ < I }. Equivalently, this plane should intersect 

V ( 2 ) 

the domain {x 2 + y 2 < z 2 }. Since this plane contains also the vertex of the cone, 
so contains a line £ C {x 2 + y 2 < z 2 } U {0}. By Lemma |iflM = 8. Therefore, 
by compactness of M, it is true for all lines close enough to £. Moreover, if I is 
contained in a plane T X M , then, due to the nonzero curvature of M at x, any line 
close enough to £ is also contained in some tangent plane to M. 



So in 4.1 applied to a small perturbation of £ the left side is equal to zero, and 
the right side is at least one, which is impossible. 

The Theorem || now follows from Lemma || applied to the restriction of the 
projection to Mr = MC\Br - intersection of M with a big enough ball Br. Indeed, 
we just proved that the projection is a local diffeomorphism. Also, the restriction 
of the projection to the boundary of Mr is diffeomorphism since the boundary of 
Mr is C 1 -close to K fl {x 2 + y 2 + z 2 = R 2 }, so is embedded by projection. □ 



5. Example 

In this section we provide an example of a hyperbolic closed connected surface 
without boundary in K 3 bounding a domain without lines inside. This surface has 
an asymptotic behavior similar to these considered above (namely it approaches 
the pseudo-cone K' at infinity), but unlike the surfaces before, its closure in M.P 3 
is not smooth and is not hyperbolic after smoothening. 

Construction starts by definition of a affine convex-concave sets. Consider a 
hyperbolic surface bounding some domain in K 3 . At each point it has a direction 
of positive sectional curvature and an orthogonal direction of negative sectional 
curvature. The affine convex-concave sets come from a requirement that these 
directions should not be far from a vertical (=parallel to z-axis) and horizontal 
(^perpendicular to z-axis) respectively. In other words, we want the horizontal 
sectional curvature to be always negative and the vertical sectional curvature to 
be always positive. The first requirements implies that the horizontal sections of 
a domain bounded by the hyperbolic surface are convex, and then the second one 
gives a concavity-type condition on these sections. We introduce the affine convex- 
concave sets as sets satisfying these two properties, i.e. using only the notion of 
convexity. This class is an affine relative of the class of L-convex-concave subsets 
of MP™ defined in [0, and is similarly closed under surgeries considered there. An 
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analogue of the first part of the Arnold conjecture can be formulated for convex- 
concave sets and for L-convex-concave sets (the second part then follows trivially) . 
We prove it to be true for the first nontrivial case of L-convex-concave sets in M , 
by a rather complicated considerations. 

The first step of our construction in this section is to construct a counterexam- 
ple to an analogue of Arnold conjecture for convex-concave subsets of R 3 . This 
counterexample is a so-called strip - a piece of a two-dimensional surface which 
is at the same time a convex-concave set. A strip in no way can be interpreted 
as a domain bounded by a hyperbolic surface. However, one can think about it 
as an interior-less limit of certain domains bounded by hyperbolic surfaces. The 
condition of absence of lines inside a convex-concave body is an open one, so any 
convex-concave body close enough to the strip also does not contains a line inside. 
The second step consists of a small perturbation of the set E - the cone K' with 
attached strip - in order to get a convex-concave set which is a genuine domain 
bounded by a hyperbolic surface. 

This perturbation is based on the fact that the class of convex-concave sets is 
closed under taking the fiberwise affine linear combinations (by Minkowsky). We 
replace E by an affine linear combination of The result is a genuine domain bounded 
by a hyperbolic surface (with an additional property that all its sections are convex) . 

5.1. Affine convex-concave sets. We will call by horizontal everything in R 3 
which is parallel to the coordinate (a;, y)-plane. For example, 

1. planes {z = c} are called horizontal, 

2. directions (a, 6,0) are called horizontal. 

Definition 4. We say that a set A C M 3 is convex- concave if 

• its sections by horizontal planes are nonempty, convex and compact and 

• the sections St = A D {z = t} depends in concave way (in Minkowski sense) 
on t. 

The second condition means that for any t\ < £2 < £3 the section St 2 is contained 
inside the linear (in Minkowski sense) combination * 2 ~* 1 St, + l 3 ~t 2 St, , i.e. the 
convex hull of the union S^ U St 3 . It can be reformulated in several possible ways. 

The first equivalent reformulation is that 

(2') for any t\ < £2 < £3 any point of the section St 2 lies on a line intersecting 
both St ± and St 3 . 

Form this follows another reformulation of the same condition. Namely, it is 
the requirement that the complement to the projection of A along any horizontal 
direction is locally convex. In other words, if we introduce a coordinate (w, z) on 
the plane of projection, then 

(2") the projection should be given by 7r(A) = {— 4>i{z) < w ^ 4>2{z)} with both 
<pi (z) and 4>2( z ) being convex and — 4>i(z) ^ <p2 (z) for all z. 

Here " f is a convex function" means that f( x+a *>~f( x ' > _ f( x )-f( x - b ) > g f or a jj 

J a b ^ 

a,b > 0. For C 2 -smooth functions this is equivalent to /" ^ 0, and for continuous 
functions it can be defined in more distribution-like spirit: / is convex if J fg"dz ^ 
for any smooth nonnegative function G with compact support or tending fast 
enough to zero as \t\ — > 00. 
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5.1.1. Support function. Let recall the definition and basic properties of the support 
functions of a convex set in R n (we will need the case of n = 2 only). Let S C R™ be 
a compact convex set. Then one can define a support function Fs(£) = max^gg £(x) 
on (R n )*. This function is clearly R + - homogeneous, i.e. Fs(ai) = aFs(£) for any 
a > and £ <= (R™)*. Since S C {h(x) «C F s (h)} H F s (^)} for any 

4,4 G (R™)*, we get that F s (a£ 1 +p£ 2 ) < aF fl (^) + 0F S {1 2 )) for any a,/3 ^ 0. 
If S" is an another compact convex subset of C R n , then Fs ^ Fs* ■ 

Vice versa, for any F : (M 2 )* — > R satisfying the previous conditions one can 
construct a convex compact figure S = <~)e e (g2y{£(x) ^ F(£)}. One can check that 
F s = F. 

If we define the Minkowski sum of two sets A C R ra and B C R™ as A + _B = 
{a + 6|a £ A,b £ B}, then the support function of A + B is equal to the sum of 
support functions of A and B. 



The boundaries of projections in the Definition |5.1| are exactly the values of the 
support function of St on this direction. Namely, if the projection is defined by 
7r : [x,y,z] -> (ax + by,z), then 0i(z) = mm^^^s^-ajx + {-b)y and (f> 2 (z) = 
max( I !( ^g Sj ax + by. So the second condition of the Definition |J means that the 
support functions of St depend concavely on t. 

We will further need the case when the boundary of projection (=support func- 
tion) is piecewise 

5.2. Strips. First we construct an unexpected object: a convex-concave set all 
horizontal section of which are segments. 

Definition5. A surface with boundary {(x, y, z) G R 3 x = u\ (z) + tf\ (z) , y = 
1*2(2) + tf2(z), \\t\\ ^ 1} which is also a convex-concave set will be called a strip. 

In other words, we parameterize the strip by two curves: one formed by the 
middle-points of the segments M = (ui(z), 1*2(2), z) and another formed by the 
ends of the segments, Mi = (u 1 (z) + /i(z), u 2 (z) + h( z )-, z ) 

An evident example of a strip corresponds to linear /1 and ji and Ui = (so- 
called degenerate strip). In this case the strip is simply a piece of a quadric and 
contains two one-parametric families of lines (one of them consists of horizontal 
lines) . So any point of a degenerate strip lies on a line intersecting all sections and 
convex-concavity follows by Definition pjj . 

It turns out that there exist non-degenerate strips, and they survive some small 
perturbations, whereas the property to contain a line doesn't. So we construct a 
strip not containing a line inside. 

5.2.1. An unperturbed strip with exactly one line inside. Consider first unperturbed 
strips, i.e. strips with Ui(z) = 0. 

We suppose further that fi(z) are two linearly independent solutions of a second 
order linear differential equation of the type y" = g(z)y. This is not very restrictive. 
Indeed, any two functions are solutions of a differential equation of second order as 
soon as their Wronskian is nonzero. But if their Wronskian is zero at z = t, then, 
assuming some genericity, its sign will change at this point. It follows (after some 
computations) that the projection along direction [fi(t), f2(t), 0) doesn't satisfy 
condition 



5.1 



Our first step is to build a strip containing exactly one line. More exact, we will 
prove that almost all unperturbed strips are like this. 
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Lemma 9. Suppose that ft, fa are linearly independent solutions of a second order 
linear differential equation of the type y" = g{z)y. Suppose that the set {x = 
tfi(z),y = tf 2 {z), ||£|| ^ 1} contains another line £ (apart from the z-axis). Then 
g(z) = 0. 

Proof. First, £ cannot be parallel or intersect the z-axis. Indeed, in this case our 
solutions are linearly dependent (since the equation of a vertical plane is y = kx) . 

So these two lines (£ and the z-axis) are not in the same plane. After a rotation 
in (x, y) plane we can assume that I £ {x — A}, so the line £ is defined by equations 
x = A,y = az + b. Consider the quotient k(z) = j^. Then fc'(z) = El^M = jL 
since W(fi,f 2 ) = const (the equation y" = g{z)y has no term with y'). From 
the other side, k(z) = s^+k j g a } mear function, so its derivative is a constant. So 
fi(z) = const and therefore g(z) = G. □ 



Lemma 10. Let fi be two linearly independent solutions of y" = g(z)y. If, g(z) ^ 
for all z € M then the set S = {x — tfi(z),y = tf%(z), \\t\\ ^ 1} is a strip (i.e. is 
convex-concave) . 

Proof. Projection of the strip in the direction (—6, a, 0) is given by ir(A) = {— \(f>(z)\ ^ 
w ^ |0(z)|}, where 4>(z) — af\(z) + 6/2(2) is again a solution of the same equa- 
tion y" — g{z)y. . We have to prove that |</>(z)| is convex, or, equivalently, that 
<j>{z)" > when <j>{z) > and that <j>(z)" < when <j>{z) < 0. But this follows 
immediately from the equation and positivity of g(z). □ 

5.2.2. Perturbation of a strip. Take any strip from the Lemma [Io| We want to 
perturb our strip between two levels in such a way that the perturbed set will be 
still a strip but will not contain any lines. There is only one line passing through 
the unperturbed part and our goal is to make sure that 

1. the perturbed part will not contain this line and 

2. that the perturbed strip will remain convex-concave. 

Here is the construction. Take any strip S described in the Lemma [l^. Take any 
p{t) such that, first, \p(z)\ ^ g(z) and, second, p(z) ^ const g(z). Take Ui(z) such 
that u'-(z) = p(z)fi(z) for i = 1,2. Consider the strip S = {(x, y, z) g E 3 | x = 
ui(z) + tfi(z),y = u 2 (z) + tf 2 (z), \\t\\ < 1}. In other words, we shift the segment 
St - the horizontal section of S - by vector (ui(t), U2(t), 0). We prove that the 
first condition on p(z) implies concave-convexity of S and the second implies that 
z-axis^ S. 



Lemma 11. S 



is convex-concave. 



Proof. The horizontal sections of S are segments, so the first condition of the Def- 
inition]!] is satisfied. We check the second condition in the form of the Defini- 



tion 5.1. A horizontal projection of S along the direction (— b, a,0) is given by 
ir(A) = {ip(z) — \4>(z)\ ^ w ^ ip(z) + \(j)(z)\}, where ^(z) = aui(z) + bu 2 {z) and 
4>{z) = ah{z) + bf 2 {z). 

We have to check that the boundary of any horizontal projection is given by 
convex functions, i.e. that 

• (V>(z) + 0(z))" > and (ip(z) - <p{z))" s$ when cp(z) ^ and that 

• (ip(z) + 4>\z))" and \ip(z) - 4>{z))" ^ when <j>(z) ^ 
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In other words, we have to prove that (<j>(z) ± tp(z))" has the same sign as <fi(z). 
This is evident since their ratio is equal to g ± p which is always nonnegative. □ 

We have to check is that the perturbation is not directed along the segments, 
i.e. that z-axis does not lie in S. 

Lemma 12. If p ^ eg for a \c\ < 1 then for some (el the section S D {z = t} of 
the perturbed strip doesn't intersect the section S fl {z — t} of the unperturbed one. 

Proof. Suppose opposite, i.e. that u% = X(z)fi(z). Then p(z)fi(z) = u" = (A" + 
Xg)fi + 2A'/j'. In other words, the vector 2\'(f' ll f! 2 ) should be proportional to 
the vector (/lj/z). This is possible if and only if A' = or the vector (f[,f£) is 
proportional to the vector (/i, f 2 ). The second possibility contradicts to the linear 
independence of f\ and f%. The first one means that A = const, so p and g are 
proportional. □ 

The perturbation can be made local, i.e. between two levels. 

Lemma 13. We can find an even p{z) satisfying all previous conditions and such 
that m{z) = for \z\ > 1/2. 

Proof. Indeed, consider the space L of even C 2 -smooth functions p(z) vanishing 
identically for \z\ ^ 1/2. The functions Ui solving u'( = p(z)fi with p S L and 
initial conditions U,(— 1) = 1) = are identical zero on z ^ — 1/2 and are 
linear on z ^ 1/2, i.e. Ui(z) — anz + aio for z > 1/2. Evidently, a^- depend linearly 
on p, so in the infinite-dimensional space L there is a subspace L', codimL' ^ 4, of 
functions corresponding to <2jj =0. □ 

5.2.3. Specification of the strip. The examples of strips constructed above depend 
essentially on g{z) and p(z) only (different choices of fi and Ui result in strips dif- 
fering by a linear transformation of R 3 ). There is a big degree of freedom in their 
choice. Here we impose some additional restrictions on these two functions and on 
the choice of fi and it^, in order to facilitate the following constructions - trans- 
forming of the strip to a domain bounded by a hyperbolic surface not containing a 
line. 

Corollary 4 (of the constructions above). Take g(z) be an even smooth function 
identically equal to for \z\ ^ 1 and strictly positive otherwise. There is an even 
nonzero function p(z) vanishing identically for \z\ ^1/2 and functions ui(z),U2(z), fi(z), f2(z) 
such that 

1. fi(z) are two linearly independent solutions of f"(z) = g(z)f(z) and u'l(z) = 
P(z)fi(z); 

2. the perturbed strip 

S = {(x,y,z)&R 3 | x = u 1 (z)+tf 1 (z), y = u 2 (z) + tf 2 (z), ||i|| < 1} 

does not contain lines and is symmetric with respect to the rotation (x, y, z) — > 
(x,-y,-z) ofM 3 . 

3. the part of the strip S lying in {\z\ 1} is bounded by four rays, and directions 
of these rays lie inside the cone z 2 > x 2 + y 2 . 

4. / 1 2 (±2) + /|(±2)<1 

Proof. Take a p(z) as in the Lemma [l3|. Take fi(z) and to be any even and 

odd correspondingly solutions of f"[z) = g(z)f(z). Then one can take an even 
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U\(z) and an odd U2{z) solutions of equations u" — p(z)fi(z). Since p(z) and g(z) 
are not proportional, the strip S does not contain lines. Together this means that 
the strip S is symmetric with respect to the rotation (x, y, z) — > (x, —y, —z) of R 3 . 

Since f"(z) = Ui(z) = for \z\ > 1, the boundary of the part of S lying 
in {\z\ > 2} is just four rays {(x,y,z)\x = ±(ao + ai\z\),y = ±&i|z|,|z| > 1}. 
Multiplying fi(z) and Ui(z) by a small number (i.e. after a dilatation of (x,y)- 
plane), we can assume that a\ + b\ < 1 and ff(±2) + /f (±2) < 1, as required. □ 

5.3. Gluing to the quasi-cone and smoothening. Here we glue the strip S of 
the Corollary [| to the quasi-cone K' = {(x, y,z) \ x 2 + y 2 = (\z\ — l) 2 , \z\ ^ 
1}. In other words, we construct a convex-concave set E with horizontal sections 
coinciding with sections of K' for \z\ 2 and with sections of S for \z\ ^ 1. 

Here is how E is constructed. Take the union E\ of S and K' . Horizontal 
sections of E\ are sometimes segments, sometimes closed discs and sometimes their 
unions. Denote by E a set whose horizontal sections are the convex hulls of the 
corresponding horizontal sections of E\. E coincides with with S for \z\ ^ 1, so in 
particular doesn't contain a line. Also, the last two conditions of the Corollary |J 
together guarantee that the part of S lying in {\z\ ^ 2} lies inside K', i.e. E 
coincide with K' outside {\z\ < 2}. 

E turns out to be a convex-concave set since its support function is a maximum of 
support function of S and a linear function - a support function of K' - overtaking 
it as z — > oo, thus convex. Here are the details. 

Lemma 14. E is a convex-concave set. 

Proof. All horizontal sections of E are nonempty and convex by definition. So we 
have to check that all projections of E are bounded by graphs of a convex functions, 



as in Definition 5.1. Let a projection of E be given by tt(E) = {—^{z) ^ w 
<j>i(z)}. We have to prove that both <j>\{z) and <p2(z) are convex. 

The proof is the same for both <f>i (z) and 4>i{z), so we consider only <f>i{z). 
Taking convex hull of sections doesn't change projection, so ir(E) = ir(E±). Let 
projections of S and K' be defined by n(S) = {— 4>§ (z) ^ w < 4>f(z)} and ir(K') = 
{-(j)f(z) ^w^ <f>f(z),\z\ > 1}. Then ^(z) = max(^f (z), <f)f (z)) for \z\ > 1 
and 4>i{ z ) = <t>i ( z ) f or \ z \ < 1- 

Let (f>^' (z) be a piece- wise linear function equal to (j>^ (z) for \z\ ^ 1 and equal 

to for |z| ^ 1. Trivially </>f'(z) is a convex function. 

Note that by choice of p(z) in the Corollary |] the middle point of the intervals 
S n{z = t} lie on the z-axis for \t\ ^ 1/2, so 4>f (z) = </>f (z) > for \z\ > 1/2. 

Therefore = max(</>f (z), (z)) for z € [1/2, oo), so is convex on this interval 

as a maximum of two convex functions. Similarly 4>\{z) is convex on (— oo, —1/2]. 
By definition <pi(z) is a convex function on [—1,1]. Therefore 4>\{z) is convex on 
the whole real line. □ 

5.4. Smoothening. The convex-concave body E built in the previous section 
doesn't contain a line but still is not a domain bounded by a hyperbolic surface. 
To finish the construction of an example we will smoothen E and will get a convex- 
concave domain D bounded by a smooth hyperbolic surface. 

The further constructions are based on the fact that even after small enough 
deformations E still doesn't contain a line. In fact, the deformation should be 
small enough only near the piece of the strip S. 
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Lemma 15. Let J?cM 3 be some set. Suppose that E' f\ {z — t} is compact for all 
t and is in an e -neighborhood of E n {z = t} for all —10 ^ t ^ 10. If e is less than 
some number c depending on E, then the set E' doesn't contain lines. 

Proof. It is enough to check only non-horizontal lines, i.e. the lines given by 
I = {(a + bz, c + dz, z)\z e R}. It is easy to see that the function maxdist(£, E) = 
maxa-g^nu^^o} dist(a;, E) achieves its nonzero minimum c on the set of all non- 
horizontal lines. Indeed, maxdist(£, E) —> oo as (a,b,c,d) — > oo, so there is 
a global minimum of maxdist(£, E). Moreover, this minimum is non-zero since 
maxdist(£, E) = would imply that t n {\z\ ^ljcfifl {\z\ <k 1} = 5" n {\z\ sC 1}, 
which is impossible. 

If e < c, then maxdist(€, E 1 ) > maxdist(£, E) — e ^ for any line I. This means 
that £ ct E'n {\z\ ^ 1}, so £ (£ E'. □ 

5.4.1. Convolution. The procedure described below are in fact a particular case of 
a general method of smoothcning of convex-concave sets: though all considerations 
are done for the set E constructed in the previous section, the constructions can 
be easily generalized for any convex-concave set with moderate growth of support 
function as \z\ — ► oo. The procedure is a generalization of the well-known fact that 
convolution of an intcgrable function with a C°°-smooth function is a C°°-smooth 
function. 

There is a well-known operation of taking an affine combination by Minkowski of 
two convex sets (by affine combination we mean a linear combination with positive 
coefficients sum of which is equal to 1). The result of this operation can be described 
in two ways: 

1. it is a set of all points which are a convex combination with the same coeffi- 
cients of a point in the first set and a point in the second set. 

2. this is a convex set with the support function equal to the linear combination 
with the same coefficients of the support functions of the first and the second 
sets. 

We can apply this operation fiberwise to convex-concave sets. 

Lemma 16. Let A and B be two convex-concave sets, and let \\ and A 2 be two 
positive numbers, \\ + A 2 = 1. Then the set C whose horizontal sections C z are 
equal to the \\A Z + \2B Z is convex-concave. 

Indeed, the sections are convex by definition and support function of C z , which 
define boundaries of projections of C, are convex in z as a sum of two convex in z 
functions - the support functions of A z and B z . 

We will apply a generalization of this operation to the set E of the previous 
section. Let take any line £ perpendicular to the horizontal planes. Consider a 
group of affine transformations of K 3 generated by translations in vertical directions 
and rotations with £ as an axis. This group T is just a cylinder r = I x S 1 : to 
(z, 4>) corresponds a composition g z < p of a shift by z along £ and rotation by angle 
(f> around I. Fix a standard Lebesgue measure fi = dzd(f> on T. 

This group acts on the class of convex-concave sets: evidently, the shifts and 
rotations around vertical axis of a convex-concave set gives again a convex-concave 
set. 

As soon as we chose £, arises a one-to-one correspondence between convex- 
concave sets and their support functions Fa(z, <):1x R 2 : for each fixed z Fa(z, £) 
is just a support function of the section A z , i.e. Fa{z,£) = max l£l 4 2 ^(1). This 
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function is convex in z, is positively homogeneous and is also convex for any fixed 
z. Vice versa, any such function defines a convex-concave set (just reconstruct each 
section separately by its support function). 

Take a <5-like function on T concentrated near the identity element of T. More 
exact, take a function K e (z, 4>) : T — > M with a following properties: 

1. J r K £ (z,(j>)dfj, = 1, and J lzlM ^ £ K e (x,(j>)dn ^ 1 - s, 

2. K E (z, </>) is C°°-smooth and strictly positive, 

3. K £ (z, 4>) and all its partial derivatives in z decrease exponentially as \z\ — > oo 

4. K e {z, (f>) is even function of z. 

For example, one can take the K e (z,(j>) = C(e)exp((cos <f> — z 2 )/e with a suitable 
choice of the constant C(e). 

We take shifts and rotations of E and define D e as an affine combination of 
the results with weight K e . In other words, we define the convex-concave set 
D e as a convolution of E with K E (x,(p): D = J r K e {z,(f))g z ^{E)dfi. Alterna- 
tively, the support function of the set D is defined by the convolution F]j(z,(j)) — 



These integrals converge since the support function of E grows as a linear func- 
tion of \z\: as soon as \z\ ^ 2, the sections of E are just circles of radius \z\ — 1. 
We claim that 

Theorem 7. 1. Sections of D e are strictly convex with nonempty interior. More- 
over, boundaries of sections of D £ are smooth and have everywhere non- 
vanishing curvature. 

2. D e is convex- concave domain. Moreover, boundaries of projections of D £ are 
smooth and have everywhere non-vanishing curvature. 

3. D E is close to E in the sense of the Lemma [7^. It means that for any 5 > 
we can find an e > such that the sections of domain D e are in the 5- 
neighborhoods of the corresponding sections of E. Therefore D e does not 
contain lines for a sufficiently small e. 

4. Boundary of D e is a smooth hyperbolic surface. Moreover, D s approaches K' 
at infinity. 

In other words, the Theorem says that the D e is an example we are looking for. 

Proof. Evidently, the support function of D s is infinitely smooth as a result of con- 
volution with an infinitely smooth function. Moreover, since the support function 
of E convexly and non-linearly depends on z, the support function of D e will be 
strictly convex in z and will have everywhere positive second derivative on z. 

To prove non-degeneracy of the curvature of the sections of D e , one should look 
on the way to restore a (planar) convex figure from its support function. Let F(£) 
be a support function of a planar convex figure P>, and suppose that F(£) is smooth. 
Then the gradient VF(£) is constant on rays beginning at the origin (i.e. is in fact a 
mapping VF : S 1 — > K 2 ), and its image is exactly the boundary of B. One can easily 
check that the boundary of B is smooth and have a nonzero curvature at Vf (fo) 



coincides with the line joining the origin and the point. 

We apply this construction to the support functions of sections of D e . The 
Hessian of the support function of a section of D e is a convolution of K e with 
the Hessian in coordinates (x, y) of the support function of E. Since the latter is 



J r K e (t, ip)F B {z -t,4>- tp)dtdi>. 
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somewhere nonzero and everywhere positively semi-definite as a quadratic form, 
the convolution will be everywhere nonzero and positive semi-definite. Thus the 
boundary of sections of D £ are smooth with nonzero curvature. 

Moreover, the gradient mapping V X! yFo s (z,£) : IxS 1 — ► dD e is a smooth 
parameterization of the boundary of D e . Taken together, this means that D £ is 
convex-concave and its boundary is smooth with everywhere nonzero curvature. 

By standard arguments one can prove that as e — > 0, the result of a convolution 
of a function Fe with K e converges to Fe itself. This implies that the set D e lies 
in a ^-neighborhood of E, as required. 

The last claim is that D £ approaches E at infinity. First, the sections E z of E are 
circles of radius \z\ — 1 for \z\ ^ 2, and therefore Fe(z,£) — (\z\ — 1)\\£\\ for \Z\ ^ 2. 
Since K e is even as a function of z, the convolution with K e do not changes linear 
functions of z. Therefore the difference \Fd s {z, £)~Fe(z, £)\ decreases exponentially 
together with all its derivatives as \z\ — > oo. Therefore the parameterizations of 
boundaries of E and D £ by the gradient of their support functions as before are 
exponentially close, which proves the claim. 

□ 
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